General Hölder Smooth Convergence Rates Follow from Specialized Rates Assuming Growth Bounds

Often in the analysis of first-order methods for both smooth and nonsmooth optimization, assuming existence a growth/error bound or KL condition facilitates much stronger convergence analysis. Hence, separate is typically needed general case growth bounded cases. We give meta-theorems deriving rates from those lower bound. Applying this simple but conceptually powerful tool to proximal point, subgradient, bundle, dual averaging, gradient descent, Frank–Wolfe universal accelerated immediately recovers their known convex optimization problems specialized rates. New results follow bundle methods, averaging Frank–Wolfe. Our can lift any rate based on Hölder continuous gradients bounds. Moreover, our theory provides proofs optimal bounds under textbook examples without